On convergence of the EM algorithmand the Gibbs sampler
نویسندگان
چکیده
SUMMARY In this article we investigate the relationship between the two popular algorithms, the EM algorithm and the Gibbs sampler. We show that the approximate rate of convergence of the Gibbs sampler by Gaussian approximation is equal to that of the corresponding EM type algorithm. This helps in implementing either of the algorithms as improvement strategies for one algorithm can be directly transported to the other. In particular, by running the EM algorithm we know approximately how many iterations are needed for convergence of the Gibbs sampler. We also obtain a result that under conditions, the EM algorithm used for nding the maximum likelihood estimates can be slower to converge than the corresponding Gibbs sampler for Bayesian inference which uses proper prior distributions. We illustrate our results in a number of realistic examples all based on the generalized linear mixed models.
منابع مشابه
Partially Collapsed Gibbs Samplers: Illustrations and Applications
Among the computationally intensive methods for fitting complex multilevel models, the Gibbs sampler is especially popular owing to its simplicity and power to effectively generate samples from a high-dimensional probability distribution. The Gibbs sampler, however, is often justifiably criticized for its sometimes slow convergence, especially when it is used to fit highly structured complex mo...
متن کاملOn convergence rates of Gibbs samplers for uniform distributionsbyGareth
We consider a Gibbs sampler applied to the uniform distribution on a bounded region R R d. We show that the convergence properties of the Gibbs sampler depend greatly on the smoothness of the boundary of R. Indeed, for suuciently smooth boundaries the sampler is uniformly ergodic, while for jagged boundaries the sampler could fail to even be geometrically ergodic.
متن کاملOn Convergence Rates of Gibbs Samplers for Uniform Distributions
We consider a Gibbs sampler applied to the uniform distribution on a bounded region R ⊆ R. We show that the convergence properties of the Gibbs sampler depend greatly on the smoothness of the boundary of R. Indeed, for sufficiently smooth boundaries the sampler is uniformly ergodic, while for jagged boundaries the sampler could fail to even be geometrically ergodic.
متن کاملRate of Convergence of the Gibbs Sampler by Gaussian Approximation
SUMMARY In this article we approximate the rate of convergence of the Gibbs sampler by a normal approximation of the target distribution. Based on this approximation, we consider many implementational issues for the Gibbs sampler, e.g., updating strategy, parameterization and blocking. We give theoretical results to justify our approximation and illustrate our methods in a number of realistic e...
متن کاملPartially Collapsed Gibbs Samplers: Theory and Methods
Ever increasing computational power along with ever more sophisticated statistical computing techniques is making it possible to fit ever more complex statistical models. Among the popular, computationally intensive methods, the Gibbs sampler (Geman and Geman, 1984) has been spotlighted because of its simplicity and power to effectively generate samples from a high-dimensional probability distr...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Statistics and Computing
دوره 9 شماره
صفحات -
تاریخ انتشار 1999